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INTRODUCTION 



According to current research, daily access to computer- 
assisted instruction for periods of ten to twenty minutes can 
increase student performance on tests. This finding has been 
replicated in several studies and reported in a recent review 



article (Kulik et al., 1983). The Kulik research synthesis 
shows a positive effect on student achievement for coraputer- 




a4 >~ . . , - , - , - - • «»«»^ig that the 

errect tends to be stronger for elementary than for secondary 
students, and stronger for tutorial than for drill and prac- 
tice programs. The synthesis performed by Kulik et al. 
showed stronger effects for low and middle ability students 
and for studies of four weeks or less in duration. 

Kulik et al. also reviewed studies of the effect of computer- 
assisted instruction on student attitudes, and found small 
but consistently positive effects. 

L i ike fno e \ arlier Education a! Testing Service study (Ragosta et 
al., 1982; the Kulik review focused on student achievement on 
examinations in traditional subject areas such as mathematics 
and language arts. However, in addition to helping students 
learn core subject material, computers are also credited with 
enhancing problem-solving skills. Though this notion appears 
self-evident to anyone who has worked with computers (speci- 
fically with children and computers), little empirical 
evidence is yet available. y 

Much of the research currently available in the area of 
problem solving has been done in relation to gifted <;duca- 
tion. However, a study of strategies of gifted children in 
solving the "Mastermind" problem (Ludlow and Woodrum, 1982) 
shows that average and gifted children used similar problem- 
solving strategies. They conclude, "Educators must develop 
curricula which include the acquisition, use, and refinement 
of higher level cognitive abilities (including problem- 
solving;. ..and design techniques and strategies for use by 
teachers in eliciting and improving these skills through a 
variety of media and content areas." (p. 103) 

A great deal of the computer use in District 181 has been in 
that semi-defined area of problem-solving. Students receive 
instruction in the logic of computer programming, and use 
software designed to stimulate their thinking. Thus, the 
Cisinct setting is one which is amenable to the 
investigation of the effectiveness of the computer as an aid 
to learning problem-solving skills. 



' 4 



BACKGROUND 



During the 1982-83 school year, a small-scale experimental 
study of using the computer to enhance problem-solving in 
mathematics was carried out in the district. A one-week unit 
on coordinate geometry was taught to fourth graders. A true 
experimental design was employed. According to Simpson 
(1983) "Schools were randomly selected to be in the experi- 
mental group in which students worked with Bumble Game 
software as part of their normal instruction in the math 
class coordinate geometry unit. In the control group 
schools, students received instruction in coordinate geometry 
as usual, with no auxiliary use of computer software." (p. 2) 

Because the posttest scores for both groups were very high, 
there was little opportunity to detect effects cf computer 
use on students' learning of coordinate geometry. The data 
analysis showed a borderline result: statistically non- 
significant when analyzed with all schools, significant when 
analyzed with Walker School data removed. Summarizing the 
findings, Simpson states: 

Students using the computers for the coordinate 
geometry uniu performed slightly higher on the 
posttest than did control group students but the 
difference in scores was slight.. .The results of 
this experiment, then, are suggestive rather than 
definitive. (1983, p. 9) 

One of the recommendations based upon the coordinate geome- 
try/problem-solving experiment was to carry out another 
experimental study under different conditions. The condi- 
tions suggested were: 1 ) develop a definition of "problem- 
solving' 1 and delineate skills and behaviors which would be 
evidence of problem-solving, 2) identify software which 
specifically addresses those skills and behaviois, 3) develop 
alternative measures of such behavior, and 4) use a longer 
experimental period. This recommendation was adopted by the 
district, and resulted in the experiment described in this 
report. 



METHOD 

The project commenced with work on the definition of problem- 
solving by a district committee. For the purpose of the 
study, "problem-solving" has been defined as skill in polving 
visual- spatial problems. The abilities to mentally manipulate 
shapes, to perceive relationships among two-dimensional 
figures, and solve spatial reasoning problems were included 
in the working definition of problem-solving. Since these 
skills are measured on most intelligence and aptitude tests, 
they are related to general intellectual performance rather 
#han to particular subject area performance. 



Nine fifth grade classes were assigned to one of three groups 



1* Computer-assisted grou p. Stuuents used computer software 
designed to help students improve spatial problem-solving 
skills for one hour per week. 

2 « Worksheet group . Students used worksheets and seatwork 
exercises designed to enhance spatial problem-solving 
skills for one hour per week. 

Control group. Students had no special instruction in 
problem-solving. 

Assignment of classes to groups was nonrandom. Teachers 
volunteered to be in the computer group or the worksheet 
group, with the remaining classes serving as control. Three 
classes were in each group. These nine classes represented 
mu fifth S rade classes at six of the seven district schools. 
The Lane School was not included because of its different 
system of grade-level organization. Figure 1 presents the 
distribution of classes and students among the three groups. 



FIGURE 1 



Group 


School 


Teacher 


Number of Students 


Computer 


Oak 

Walker 
Prospect 


De Bickero 

Otto 

Tupy 


25 
14 
17 


Worksheet 


Madison 
Elm 

Monroe 


Hummer 

Massura 

Ranck 


27 
32 
21 


Control 


Prospect 
Oak 

Monroe 


Gelvicks 
Webb 

Boyleston 


17 
23 
21 



TOTAL 1 98 

Since time spent actively engaged in learning is an important 
variable in any study of achievement, it is a variable which 
must be held constant in all groups which are receiving a 
treatment. Thus, teachers of the computer and worksheet 
groups were asked to keep log sheets of the actual time spent 
by each individual (for computer group) or the entire class 
(for worksheet group) on the visual-spatial instruction. In 
preparation for the experiment, a meeting was held for all 
fifth grade teachers from the six schools involved. The 
project description and instructions for implementing the 
different treatments were distributed and discussed. The 
materials are reproduced in Appendix A 



Teachers of the computer group were instructed that their 
students were M to work solely with the computer software." 
Thus little or no teacher instruction and use of 
supplementary worksheets occurred in these groups. The 
instruction was basically provided by the computer programs 

&X0M6 • 



Teachers of the worksheet groups were given packets of work- 
sheets numbered in the sequence in which they were to be 
used. They were to to plan lessons around these using the 
following steps: "1. Teacher explanation of main idea of the 
lesson, 2. Working through one or two examples with whole 
class, 3. Time for students to work individually on several 
examples OR group work." 

Control group teachers were asked only to refrain from using 
the problem-solving software and the worksheets during the 
study. ° 

All students were pre- and posttested with the following 
instruments: New Jersey Test of Reasoning Skills , Sequences 
and Analogies subtests of the Test of Cognitive Skills , and a 
seven-item spatial test (non-commercial j. A Cognitive Skills 
Index (IQ score) for each student obtained in fourth grade 
was also used in the analysis. 

Students were pretested during the week of December 12, 1983. 
Instruction commenced in the computer and worksheet groups 
the first week of January, 1984, and continued through the 
last week of February for a total of eight weeks. Posttest- 
Ing occurred the week of March 5. 

A debriefing meeting for fifth grade teachers was conducted 
by Ann Dana, District 181 Computer Consultant, in April 
following all posttesting, but preceding analysis and 
reporting. Teachers were invited to comment upon any aspect 
of the study and to ask questions. They were also ansured 
that they would be informed of the results of the study when 
those were available. 
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DESCRIPTION OP MATERIALS AND TESTS 



Software 

Most students in the computer groups used four software 
packages at some time during the eight weeks of instruction. 
Two software packages were available for the first few weeks 
of January, "Flip Plop" and "The Factory," both produced by 
Sunburst. 

According to the publisher's description, "Flip Plop" is 
designed for students in grades four through six. The 
purpose is to provide practice in visual discrimination 
through two "games." The first game requires identification 
of a figure which is the same as a given figure, but rfhich 
may have been either rotated or flipped. The second game 
requires the student to manipulate a second figure until it 
appears in the same orientation as the given figure. Both 
games offer ten problems at each of four levels. Since the 
version of "Flip Plop" used was pr epublicat ion, a few bugs 
remained in the software and sometimes interfered with stu- 
dents' use of it. 

"The Factory" is intended for use by students in grades four 
and up. According to Sunburst, "The program challenges 
Btudents to create geometric 'products' on a simulated 
machine assembly line" whose machines they design. Students 
must use inductive thinking, visual discrimination, spatial 
perception, and logic to succeed in the tasks. Students are 
stimulated to use problem-solving strategies such as working 
backwards, analyzing a process, determining sequence, 
applying creativity, and seeking a more efficient solution. 
Students are introduced to "The Factory" by testing the 
effects of various machines. Then they build their own 
assembly line and watch the results. Finally they are 
challenged to build an assembly line which will duplicate a 
given product. This software package won the "Best Software 
of the Year 1983" award given by Learnin g magazine. 

During the latter half of the instructional period, two 
additional software packages became available for use with 
students. These were "Pun House Maze" and "Jigsaw 
Jenerator." The former is another in the Sunburst series 
"Strategies in Problem Solving" designed for use in fourth 
grade and above. Students aim to discover the repeating 
pattern of turns which will lead them out of a three-dimen- 
sional maze. They can either guess the pattern itself, or 
apply it as they move through the maze to the exit. The 
patterns are randomly generated by the computer. The 
program's teaching objectives are "1. To provide practice in 
pattern recognition and identifying multiple solutions, 2. To 
build skills in the problem solving strategies." 
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Jigsaw Jenerator" is produced by L & S Software of Chicago, 
students are challenged to assemble puzzles of ten to forty- 
nine pieces on the screen. The pieces must be rotated and 
the positions matched carefully before correct placement will 
be accomplished. The player is offered options concerning 
the picture formed and the number of pieces in the puzzle. 

Worksheets 

Most of the worksheets used were gleaned by the research 
consultant from a Barron series book on preparing for tests 
of various kinds. The title of the section in which the 
pages were found was "Mechanical Aptitude and Spatial 
Relations Tests." The pages were intended to be used as 
practice for taking such tests. 

The microcomputer consultant also located a set of worksheets 
called Figure Relations (B) in gifted education materials by 
Midwest Publications. These worksheets were provided to the 
teachers. Brief tips for use were included with the work- 
sheets, as well as an indication of the sequence in which 
they were to be used. No answer keys or specific lesson 
plans were provided. 

Tests 

The New Jersey Test of Reasoning Skills is a fifty-item 
mult.iple-choice paper and pencil test of elementary reasoning 
and inquiry skills. It was developed by Dr. Virginia 
Shipman, Senior Research Psychologist with Educational 
.Testing Service. The items on the test ar* designed to 
concentrate on logical reasoning, and therefore items drawing 
on other skills such as vocabulary and computation have been 
avoided. The reading level is reported as being between 4.5 
and 5.0. The youngest level for recommended use is fifth 
grade. The reported reliability is .84 which is quite 
adequate for research use. Examples of skil.'.s tested are 
part-whole reasoning, discerning causal relationships, 
identifying good reasons, syllogistic reasoning, 
distinguishing differences of kind and degree, tnd detecting 
assumptions. The author states that the same form of the 
test may oe used for both pre- and post-testing if both are 
done in the same year. 

The Test of Cognitive Skills (TCS) is an abilitv test 
designed to assess a student T s academic aptitude, "it is 
published by CTB/McGraw Hill. Each level of the TCS includes 
four subtests: Sequences, Analogies, Memory, and Verbal 
Reasoning. Level 2, for grades three through five, was used 
in the problem-solving study, given as a pretest at the end 
of grade four and as a posttest immediately following the 
instruction. The Cognitive Skills Index V CSI) is an IQ-like 
score computed from a student's score on the four subtests 
and his/her chronological age. It is a normalized standard 
score with a mean of 100 and standard deviation of sixteen. 
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In addition to the CSI, scores on two of the subtests were 
used in the data analysis for the present study. The 
Sequences and Analogies subtests were chosen because they 
were most like the problem-solving instruction in that they 
are presented largely through figures rather than words, 
They do not, however, require mental manipulation of these 
figures in space, as will be seen in the descriptions below. 

The Sequences subtest contains twenty items designed to 
measure the student's ability to recognize a pattern in a 
sequence of figures, and to extrapolate to a rule. Some of 
the patterns used are progress ional sequences and 
combinations of parts to form a whole. Pour items involve 
rotation or flipping as part of the sequence. 

The Analogies subtest contains twenty items that measure the 
student's ability to recognize literal or symbolic 
relationships. These items reflect such "problem-solving" 
tasks as comparing, perceiving the function of an object, 
understanding degree or proportion, and recognizing a 
numerical, quantitative, or spatial relationship, of the 
twenty items, three required the type of visual-spatial 
skills practiced by the students during instruction. 

The seven-item spatial test was found in a Barron's series 
book on preparing for test-taking. The test is titled 
Spatial Views Test I." Students are given separate top, 
side, and front views of aw object, then asked to pick its 
three-dimensional representation. 



DESIGN 

A quasi-experimental design was chosen. This differs from a 
true experimental design in that selection and assignment of 
classes to treatments was not random. This design does 
however provide most of the same controls that an 
experimental design does, dependent on the extent to which 
the classes assigned to the various treatments are similar. 
In addition, a statistical adjustment for some of the pre- 
existing differences among classes can be made through use of 
covariates in the data analysis. 

Factors such as instructional time, materials used, and 
time of year were controlled through the instructions to 
teachers. The factor of outside opportunities to learn, 
either prior to or during the experiment, was controlled by 
the selection of visual -spatial manipulations as- the topic of 
study. This topic is not commonly taught in or out of 
school at the fifth grade level. 

Multivariate analysis of covariance was used to analyze the 
data. "Multivariate" means that multiple dependent variables 
(posttests) may be considered simultaneously. "Covariancr " 
means that posttest scores will be adjusted for the effects 
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of other related variables in order to "equalize" the groups. 
The covariates employed in this analysis were IQ and pretest 
scores or. the New Jersey test and the spatial test. The 
dependent variables were posttest scores on the New Jersey 
test, the spatial test, and the Sequences and Analogies 
subtests. The independent factors were treatment group, sex. 
and school. Sex and school were included as control factors 
txnee both are potential causes of variation in posttest 
scoree. If they are controlled by being included in the 
design, their effects will not interfere with detection of 
variation in posttest scores caused by assignment to a 
treatment group, which is the primary factor of interest. 



REPLICATION OP STUDY 

Two maladies commonly affect educational research. The first 
is obtaining nonsignificant results due to not using a large 
r™fj„ 8amp i le * Siz Jv , The second ^ obtaining significant 

f J ?n ^ 0 -l ly * t0 fl ^ d that no one else cpn replicate them. 
In an effort to avoid both of these situations, District 181 
secured the cooperation of another district in replicating 
f U n y * e °° pl . bed in this report. The Cupertino Union 
School District in California shares an active interest in 
the use of microcomputers in schools, yet represents a dif- 
ferent setting in terms of both student population and 
program. A reciprocal agreement was devised according to 
which District 181 replicated a computer literacy study 
originally carried out in Cupertino, in exchange for their 
cooperation in replicating the problem-solving experiment. 

Three fifth grade classes, each assigned to one of the 
groups described above, are involved in the Cupertino study. 
Their eight-week period of instruction occurred during the 
months of April and May, 1984. Students were given the same 
tests as the District #181 students, and used exactly the 
same software and worksheets. 

When the results of the Cupertino study become available, 
they will be analyzed in combination with the District #181 
rssux xs • 



RESULTS 

Time Logs 

Teachers of the computer group were asked to keep records of 
txme spent with the visual-spatial software for each 
individual student in the class. Two teachers handed in very 
complete logs for their classes. 

The first leg showed that for six of the eight weeks nearly 
every student (absent students excepted) spent three twenty- 
minute cessions at the computer with a partner. The other 
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two weeks were disrupted by field trips, hqg*idays, etc. and 
only two twenty-minute sessions were held. Records indicated 
that no one student missed more than three sessions due to 
absence • 

The second complete log indicated that each student spent 
sixty minutes each week for eight weeks using the problem 
so) ving-sof tware. Most of that time was spent in pairs at 
the computer, though some was spent in triples and some spent 
alone. 

The third computer group teacher described for the class as a 
whole the procedures that were used. She stated that 
students in her room were on the computer in pairs for thirty 
minutes twice a week for the first four weeks, and in triples 
twice a week for thirty minutes for the last four weeks of 
the study. She did not indicate deviations from this for 
particular weeks or particular students. 

One worksheet teacher indicated on her time log that she 
spent twenty minutes three times per week for for seven weeks 
of the study, with two twenty- minute sessions the eighth 
week. These were whole-class sessions. Absences of 
individual students were recorded; no one student missed more 
than three sessions. No make up lessons were given. 

The second worksheet teacher indicated teaching sixty minutes 
total for each of six weeks, three times in two sessions of 
thirty minutes and three times in three sessions of twenty 
minutes. One of the other two weeks had seventy minutes and 
the other had forty minutes of instruction. No student 
missed more than three sessions, and lessons were generally 
not made up. 

The third worksheet teacher indicated an irregular pattern of 
scheduling for lessons. Only seven weeks included instruc- 
tion since the first week was devoted to "testing" and the 
week of February 13 missed due to Outdoor Education. The 
number of minutes per week ranged from forty-five to sixty- 
five and lengths of individual sessions ranged from ten to 
thirty-five minutes. 

* Monitoring Visits 

Thirteen visits were made to computer group classrooms over 
the eight weeks of the instruction, and twelve visits to 
worksheet classrooms. Visit3 were made by the district 
microcomputer consultant, the IER research consultant, and a 
microcomputer consultant from IER. Most visits included 
observations of the students learning visual-spatial skills 
either on the computer or with worksheets. A few visits 
consisted only of conversations with the teacher regarding 
her implementation of the treatment and the students 1 
reactions to it. 
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Students in thecomputer group were observed using Factory 
(four times), Flip Flop (one time), Fun House Maze (four 
times), and Jigsaw Jenerator (two times). In only one case 
did 8 session include a teacher giving instruction to a group 
about how to use a program; in all other visits the students 
were using the computer independently. No observation was 
made of records being kept of student performance on a 
program* Students working together in pairs or triples were 
^?nerally observed to do so well, with all members of each 
, oup actively participating whether at the keyboard or not. 
In at least one class, students sitting near the computer, 
but not supposed to be working on it, also watched and gave 
suggestions to the student on the computer. 

Several teachers made corements on the programs cr instruction 
during the visits. They inquired about the availability of 
other software (besides the first two packages) and told 
about problems with opera, ting the various pro grains. One 
commented on the improvement in group relatio> 3 resulting 
from the experience of working together on t v a computer. 
Another questioned the validity of the tests, being used 
because they were "word tests" while the tasks on the 
computer used mainly figures. Several said the students 
really enjoyed working with the programs but tended to get 
bored after two weeks or so with the same program. 

Eight of the twelve visits to worksheet teachers incorporated 
observations of instructional sessions. All worksheet 
sessions involved some sort of class discussion led by the 
teacher and centered on a worksheet exercise. About a fourth 
of the sessions involved checking of answers students had 
worked out previously. Most of the sessions consisted of 
group work rather than individual work. Because of the lack 
of an answer key, there was sometimes confusion among 
teachers as well as students about correct answers. Also, on 
at least two occasions, i* was observed that teachers had 
seme difficulty in expressing why incorrect answers were 
incorrect. 

One teacher had made overhead transparencies of all the 
figures used on the worksheets so that they could be flipped 
and rotated as the students Wctched. 

Some of the worksheet teachers made comments regarding the 
study procedures and materials during the monitoring visits. 
Several said that their students enjoyed the worksheets and 
found them challenging. One teacher Laid that the 
instruction took a lo* of time from other school subjects. 

Debriefing Meeting 

Two of the three worksheet teachers and all three of the 
computer teachers attended a delriefing meeting on April 11* 
The microcomputer consultant led the meeting and recorded the 
teachers 1 comments. Teachers were invited to share their 
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reactions to the procedures used in the study, the materials, 
the design, and any other aspect about which they had 
opinions. 

With respect to procedures, all teachers but one stated that 
they had used only class time for the visual-spatial 
instruction. The other teacher had scheduled some students 
at lunch time in order to fit everyone in. The others fit 
the computer and worksheet activities in by leaving out 
activities with other software which they might otherwise 
have used in conjunction with specific subjects. 

Some teachers felt that too much time was taker away from the 
academic areas. They suggested doing such a study within the 
subject-matter areas as not to cause this problem. 

The teachers generally agreed that the time period chosen for 
the study was poor because it occurred during a time which 
they regarded as being optimal for them to accomplish much of 
the regular curriculum. They suggested a bef ore-Thanksgiving 
time as an alternative. 

The teachers remarked that the tests given pre and post did 
not test what had been practiced by the students. The 
computer teachers particularly felt that the dynamic skills 
used in the computer instruction woui^ not transfer well to 
written tests. The teachers also said that more specifically 
visual-spatial problems re needed. 

Several teachers reported that they had felt nervous when 
their classes had been visited by the research and 
microcomputer consultants from IER. 

The two worksheet teachers felt there ha* not been enough 
structure for the worksheet groups in terms of how to deliver 
or how to schedule the instruction. 

Several computer teachers spoke of difficulties in scheduling 
computers for use by their students during the study. 
Regarding the software used, they felt "Jigsaw Jenerator" was 
the most challenging, and "Flip Plop" the easiest. They also 
had all found at least one flaw in the "Flip Plop" program. 

Tests 

Although data were collected from 198 fifth graders, 54 of 
these cases were missing one or more pieces of information. 
Thus 144 cases were usable in the statistical analysis. 

Figure 2 presents the means for each of the treatment groups 
on each of the (pretests and) posttests. The means used have 
been adjusted for the effects of the covariates. (Observed 
means for each class are reported in Appendix B.) Means for 
the New Jersey test and the spatial test are in raw score 
units. Maximum scores for these tests were fifty and seven, 
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respectively. Means for the Sequences and Analogies tests 
are in scaled score units. Maximum scores were 713 and 703, 
respectively. 



FIGURE 2 



Test 

New Jeioey 
Spatial 
Sequences 
Analog:, es 



Computer 
(35.72) 37.11 
(6.15) 6.33 
(550.38) 637.28 
(540.14) 644.59 



GROUP 
Worksheet 
(36.02) 37.18 
'6.04) 6.46 
(526.23) 608.99 
(526.00) 629.64 



Control 
(35.51 ) 37.54 
(6.14) 6.46 
(542.02) 636.62 
(534.39) 655.98 



Multivariate tests of significance were performed for each of 
the factors in the design: group, sex, and school. The 
tests indicated that the effects of group and sex were 
nonsignificant. Nonsignificant means that the variations in 
the posttest scores from computer group to worksheet group to 
control group, and from male to female, were so small that 
the most reasonable explanation was chance variation. The 
effect of schools, on the other hand, was statistically 
significant. 

Statistical significance, which is that level at which the 
"chance variation" explanation becomes untenable, is commonly 
defined as occurring when the probability value associated 
with the statistical test is smaller than .05. The probabil- 
ity values for the tests of all three effects are presented 
in figure 3. 



FIGURE 3 



Effect 

Group 

School 

Sex 



Probability 

.362 
.008 
.408 



DISCUSSION 

The absence of a significant effect for either the computer 
group or the worksheet group over the control group may be 
attributed to a variety of factors. The present research 
does not tell us which one or combination of these is in fact 
the cause, but may allow us to judge which are most reason- 
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able in the light of the other results of the study (time 
logs, monitoring visits, and debriefing). 

The fact that the instructed groups (computer and worksheet) 
did not show any greater achievement than the non-instructed 
group, while all had consistent patterns of gain from pre- to 
post-test (see Appendix B), strongly suggests that the 
testing detected a maturation effect, which would of course 
be common to all fifth graders. Recall that the teachers had 
criticized the tests as not appropriate for the skills 
taught. These results seem to suggest that what was tested 
was the development of broad reasoning skills which were not 
affected by the acquisition of specific visual-spatial 
skills. 

The definition of problem-solving as visual-spatial skill 
focused on a narrow area, but underlying the selection of the 
tests was the assumption that such skills would transfer to 
more general thinking and reasoning skills, particularly 
those that used a figural medium. This assumption may have 
been incorrect. The one test which was most closely related 
to the tasks students practiced was the seven-item spatial 
test. However, pretest scores on this instrument were so 
high that students had no opportunity to display further 
achievement on this type of items. 

Another possible cause of the nonsignificant effects is the 
lack of consistent treatments over the course of the study. 
However, the monitoring visits and the logs indicate that for 
the most part the treatment was given as specified. 

The use of the computer software as stand-alone instruction 
is a third possible source of the non-significant results. 
Sunburst Communications, the publisher of th^ee of the pro- 
grams used, has developed a comprehensive problem-solving 
skills instructional strategy. The strategy is based on a 
problem solving skill matrix, the components of which are 
memory, cognitive skills, cognitive control strategies, and 
creativity. They believe these to be hierarchical in nature, 
and have thus devised their activities to proceed 
sequentially from one level to the next. The curriculum they 
propose integrates problem-solving skills software w ith 
classroom activities. The software is "used to reinforce 
learning and make it fun, 11 while the classroom activities are 
used to introduce and teach directly the problem solving 
skills. In the present study, programs were used in a 
"stand-alone" manner as the primary means of instruction: 
not only is this contrary to the Sunburst philosophy, it is 
also an unrealistic pattern of use for a classroom teacher. 
Different results may have been obtained h*H. classroom 
activities and teacher instruction been incor porated into a 
planned sequence of software use. 
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SUMMARY 



This project studied the effect of use of problem-solving 
computer software on student performance on tests of 
reasoning and problem-solving ability. This performance was 
compared with that of students learning the same skills 
through use of worksheets, and with students receiving no 
specific instruction in these skills. A quasi-experimental 
design was used w ith teachers volunteering to be in the 
"computer" group, the "worksheet" group, or the control 
group. 

For purposes of the study, prohlem-solving was defined as 
being skill in manipulating figures in two-dimensional space 
(visual-spatial skill). The software and worksheets were 
selected with this specific area in mind. 

Nine fifth grade classes participated in the experiment, with 
three being assigned to each condition. Meetings were held 
with teachers prior to the beginning of the eight-week 
instructional period to insure that all teachers used similar 
tea aing methods. Observation visits were made during the 
eight weeks to further insure that this was happening. 

Each student in a computer or worksheet group received 
approximately sixty minutes per week of instruction. This 
was confirmed by the time logs kept in each class. Computer 
time was generally spent by two or three students working 
together at one computer. Worksheet instructional time wes 
generally spent in whole-class activities. 

The analysis of the test data showed that there were no 
differences in achievement between any of the three groups. 
Each group did, however, make gains on each test from 
pretesting to posttesting. 

These results strongly suggest that a common factor, such as 
maturation or normal intellectual growth, was measured by the 
tests used. Though students may have made specific visual- 
spatial skill gains, these were not be detected by the 
instruments used. Both the comments of the teachers regard- 
ing the inappropr lateness of the test instruments and an 
item-by-item content analysis of each test confirm that three 
of . the four instruments addressed general reasoning skill 
with only negligible opportunity for students to use visual- 
spatial manipulations. The fourth test did use problems very 
much like those the students practiced in instruction, but 
pretest scores on this test were so high that it was not 
possible for students to demonstrate any significant gains on 
the posttest. 

It is also possible that the use of the computer programs 
as the primary means of instruction was less effective than 
using them as a supplement to teacher instruction. Sunburst, 
the publisher of three of the programs used, advocates an 
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integrated approach to problem-solving instruction wherein 
the teacher introduces certain skills which are reinforced 
through use of the computer programs. Students especially 
need to be shown how to make the connection between the 
strategies they devise in order to solve a specific problem, 
and more generalized problem-solving and reasoning skills. 
Research on learning has shown that such a transfer does not 
occur automatically for most students: it must be pointed 
out by a teacher. 

Options for Further Study 

New research questions arise as a result of this study which 
may suggest fruitful directions for future research in 
District #181. After consultation with the district micro- 
computer coordinator and microcomputer consultant, the 
following options are presented. 

1. Study problem-solving skill development in connection 
with the use of LOGO. A review of Seymour Pappert's 
theories should be the foundation for the study. Ways 
of using LOGO would need to be standardized to minimize 
variation due to the knowledge and skill of particular 
teachers in using this software. Careful attention 
should be paid to selection of appropriate tests. A 
similar experimental design could be employed, perhaps at 
a lower grade level, over a four to eight week period. 

2. Focus attention on subject-specific software. Research 
shows that simulation software in particular helps to 
increase student scores on tests over subject matter they 
have been taught. Since both integration of computer j 
into the regular curriculum and the area of science are 
of current interest in District #181, this combination 
might provide the setting for a very interesting investi- 
gation. This project would require the surveying of 
currently available science software and comparing it 
with district science objectives. Again, selection or 
creation of appropriate tests is also an important 
consideration. 

The purpose of the experiment reported above was to determine 
whether the computer could be used to enhance development of 
problem-solving skills. Such enhancement was not 
demonstrated. However, certain factors such as 
appropriateness of tests and of teaching methods might have 
influenced the findings. This realization will be valuable in 
designing future research studies of the role of computers in 
schools, not only in District #181 but across the nation. 
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PROBLEM SOLVING PROJECT DESCRIPTION 

Memo to : Fifth Grade Teachers 

From : Ann Dana and Nan Simpson 

Re : Computer Problem Solving Project 

by WnST^t2&? H StUdyi ' n9 thG ac « u1sitio " of Problem solving skills 

For the prupose of the study, "problem solving" has been defined as skill in cniwinn 
visua -spatial problems The ability to mentally manipulate shape? to JerceiJe 9 
^Jw^^'TS J w ?- diTOnsl *o^l figures, and solve spatial reasoning 
problems are included in our working definition of problem solving These skills 
are among those measured on nearly all IQ and aptitude tests , so the studv 
is an important and interesting one! y 

All district fifth grade classes have been been assigned to one of three groups: 

^ &^ aS ! iS - ed group - students will use computer software designed to 
help students improve spatial problem-solving skills for one ho5r per week. 

2 * So^n!! e f . 9r0U E* Studen * s will use worksheets and do seatwork exercises 
designed to enhance spatial problem-solving skills for one hour per wlek. 

3. Control group. Students will have no special instruction in problem solving. 

« L fof?ows: S (C0!npUter ' Works heet, and Control groups) will be pre and posttested 

Pretests: New Jersey Test of Reasoning 

Wheatley Test of spatial relationships (or similar instrument) 

Posttests: Same 2 tests as pretest AND 
Test of Cognitive Skills (TCS) 

The schedule for the project is: 

Week of December 12 Students are pretested (both tests). Total time required 

is about 1% hours. 

Jan. 12 - March 1 study in progress. All students in computer and worksheet 

groups devote one hour/week to problem solving activities 
(Separate sheet for each group gives more detail). 

Ifardl 1 c! U ?? n i tS „? re !?? stt ? s ted with N.J. Test of Reasoning and 

Spatial Visualization test, (requires Vt hours). " 

S^b^l^ci;!)' 6 5tUdCntS POS " eSted w1th TCS ,cve ' 2 <"* uir « 1 "»«■•) 
"^tested? " St StUdC " tS ° n datos Msted - " ote AU 5th 9 rade 
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Memo'to: Teachers of Computer Group 

From : Ann Dana and Nan Simpson 

Re : Computer Problem Solving project 

Before reading this, please be sure to read the PROBLEM SOLVING PROJECT DESCRIPTION. 

o^nr.^"!- *"JV pen . d 222- iioyx £er week for eight weeks using special computer 
programs designed to enhance spatlaTprobTan soTVUig" ikiTTs. Arrangements have 
been made for you to have priority use of the school 's aval able cSerl 

S^fSff^/r an V our sch00] ' s LRC dire ctor to work ou?7schedule. 
The schedule should be completed before Christmas vacation. 

a? C thp t !JS^ h0Uld haV t th : ee 20 minute se "ions or four 15 minute sesrions 
at the computer per week. You may group students if necessary, but there should 

5; "? l 3& l t *" l ?I ee P r t our stents in a group. Two or three per o™ P is 
Ideal, with students sharing responsibility for interacting with the computer. 

sJfJ!liL tlB 1: ^/xpect the computer group to work solely with the computer 
InM? A Vi e exDen,n ent proceeds we'll be in touctwith you to see if 
auxiliary worksheets are needed. y iee 

Two software packages are now available: FACTORY and JIGSAW. We have ordered 
S°a°W fS SE ** FUP FL0P * A11 stre ss spatiai visual IzaUonlJil Is . 

to^<f n ?/ ni n"5 e ?i! le t1 *» Vl h y . 0U t0 preview the software and plan ways 
to use it. Once the project begins, you will be asked to keep a log of 

J ?? hp e n^?/S a J ial ^ ki11 devel0 . Dment ^ each of your students. A form 
win oe provided for the purpose. 

Ann Dana and Nan Simpson (our research consultant) will be in touch with vou 
as the study gets underway. If you have questions, contact Ann * 

Hhenuay R *"* ReSUltS Wil1 be Shared w1th a11 teachers Participating 



ERIC 22 



To : Teachers of Worksheet Group 

From : Ann Dar.? and Nan Simpson 

Re : Computer Problem Solving project 

Before reading this, please be sure to read the PROBLEM SOLVING PROJECT DESCRIPTION. 

Your students will spend one hour per week for ei ght weeks doing special 
exercises and worksheets to help tfiein improve their spatial visualization 
skills. Please try to schedule the hour per week as three 20 minute sessions 
or two 30 minute sessions. You will be asked to keep a log of time expended 
which specifies the amount of time spent by each child. A log form will be 
provided for this purpose. 

You will receive 24 sets of "lessons" or worksheets. The worksheet sets are 
numbered and should be used in this order. The 24 sets of worksheets will 
suffice for three sessions/weed for 8 weeks if you use one set per lesson. If 
you plan two half-hour sessions you'll need to use about \h sets per lesspn. 

Each set of worksheets is accompanied by brief tips for using. You need not follow 
the tips exactly. However, please try to include in each lesson the following 
steps: 

1. Teacher explanation of main idea of the lesson ("looking for differences"; 
"looking for patterns", etc.) 

2. Working through one or two examples with whole class 

3. Time for students to work individually on several examples 
and/or 

' 4. Group work. This could be class discussion, or a short exercise in which 
pairs of students each construct an exercise for the partner to solve. 
An alternative exercise for some lessons might te to practice with cutout 
shapes. 

Be sure to spend the designated one hour per week on these activities! Very 
little information has been published regarding the effectiveness to teach 
spatial problem solving. Your work with your students is an important part of the 
study. 

Ann Dana ind Nan Simpson (our research consultant) will be in touch with you 
as the study gets underway. If you have questions, contact Ann. 

THANK YOU FOR YOUR HELP! Results will be shared with all teachers 
participating in the study. 
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Memo to : Teachers of -Control Group 

.From : Ann Dana and Nan Simpsor 

.Re : Computer Problem Solving project 

Before reading this, please be sure to read the PROBLEM SOLVING PROJECT DESCRIPTION. 

Your students are in the control group. This means they will have no special 
problem soiving instruction for the eight weeks beginning January 3. 

Since your students will provide important comparison data, they will be 
pretested and postle>ted according to the schedule given in the Problem 
Solving Project Description. 

After March 1, if you like, you will be free to use the problem solving software 
and/or the worksheets. Also, you will receive results of the study once the 
data analysis is complete. 

THANK YOU FOR YOUR HELP! 
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PROBLEM SOLVING LOG 
WORKSHEET GROUP 



This log form is to record all problem-solving sessions you conduct with your class. 
There is a 'space representing each day of the week for each week of the problem- 
solving study. Complete the log by filling in the number of minutes your class 
spends on problem-solving worksheets and exercises for the appropriate days. 

Son. students may be absent on the day the worksheets are used. Please write on the 
lines at the bottom (continuing onto another sheet if needed) the names of students 
missing problem-solving lessons, the date of the missed lesson, and whether the 
student received a make-up lesson, (It is not necessary for you to give a make-up 
lesson, but we need to know for each student the total amount of time spent on 
problem solving activities). 



1 . .MOM... . 1 . .TUES. . 1 . .WED. . . 1 .TUURS. . 1 . .ESI. . . ! 



WEEK OF 



Jan 3 












Jan 9 












Jan lfi 












Jan 23 












Jan 30 












Feb 6 












Feb 13 












Feb 20 












Feb 27 













Students missing lessons 
Date Name 



Made up 
lesson? 



Date 



Name 



Made up 
lesson? 
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Continue on separate sheet If needed 



PROBLEM SOLVINb LOG SHEET 
COMPUTER GROUP 



It is important to know how much time was spent bv each of your students on 
problem-solving activities. Please complete one log form each week (we'll supply 
xeroxed copies with your class list to save your time). Each time a student spends 
time with the problem-solving software, write the number of minutes in the appropriate 
day-specific space, in parentheses, write the number of students working together 
if students worked in small groups. An entry night have the form 20(3) for example 
if -a group of 3 students had a 20 minute session at the computer. 



WEEK BEGINNING 



DAY 

.! HON ! TUES ! WED ! THURS ! FRI 



STUDENT 



APPENDIX B 



27 



ERIC 



24 



OBSERVED SCORES FOR COMPUTER, WORKSHEET, AND CONTROL CLASSES 

Tests 



Computer 
Classes 



Worksheet 
Classes 



New Jersey Spatial Sequences Analogies 
p re post pre post pre* post pre* post 



1- 37.12 39-85 6.2 6.7 541 653 563 678 

2. 33-42 34.78 6.0 6.3 541 600 V93 542 

3- 36.12 37.52 6.2 6.2 562 642 551 669 



1- 33-25 34.78 6.2 6.1 526 587 526 607 

2. 38.15 38.48 6.2 6.5 521 610 526 603 

3- 37.47 38.50 5-6 6.4 533 613 526 632 

Control 
Classes 

1- 37.71 38.09 6.4 6.4 535 640 557 641 

2. 36.65 40.65 6.2 6.6 557 648 53^ 633 

3- 32.43 34.92 5.7 6.4 533 617 508 661 

* estimated from percentile ranks 
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